import { Callout } from "nextra/components";
import { snippets } from "@/lib/generated/snippets";
import { Snippet } from "@/components/code";

# OpenTelemetry

<Callout type="info" emoji="🪓">
  OpenTelemetry support is currently only available for the Python SDK.
</Callout>

Hatchet supports exporting traces from your tasks to an [OpenTelemetry Collector](https://opentelemetry.io/docs/collector/) to improve visibility into your Hatchet tasks.

## Usage

### Setup

Hatchet's SDK provides an instrumentor that auto-instruments Hatchet code if you opt in. Setup is straightforward:

First, install the `otel` extra with (e.g.) `pip install hatchet-sdk[otel]`. Then, import the instrumentor:

<Snippet
  src={
    snippets.python.opentelemetry_instrumentation.worker
      .configure_the_instrumentor
  }
/>

You bring your own trace provider and plug it into the `HatchetInstrumentor`, call `instrument`, and that's it!

<Callout type="info" emoji="🔭">
  Check out the [OpenTelemetry
  documentation](https://opentelemetry.io/docs/languages/python/instrumentation/)
  for more information on how to set up a trace provider.
</Callout>

### Spans

By default, Hatchet creates spans at the following points in the lifecycle of a task run:

1. When a trigger is run on the client side, e.g. `run()` or `push()` is called.
2. When a worker handles a task event, such as starting to run the task or cancelling the task

In addition, you'll get a handful of attributes set (prefixed by `hatchet.`) on the task run events, such as the task name and the worker ID, as well as success/failure states, and so on.

Some other important notes:

1. The instrumentor will automatically propagate the trace context between task runs, so if you spawn a task from another task, the child will correctly show up as a child of its parent in the trace waterfall.
2. You can exclude specific attributes from being attached to spans by providing the `otel` configuration option on the `ClientConfig` and passing a list of `excluded_attributes`, which come from [this list](https://github.com/hatchet-dev/hatchet/blob/main/sdks/python/hatchet_sdk/utils/opentelemetry.py).

## Integrations

Hatchet's instrumentor is easy to integrate with a number of third-party tracing tools.

### Langfuse

For example, you might be interested in using [Langfuse](https://langfuse.com/) for tracing an LLM-intensive application.

<Callout type="warning" emoji="⚠️">
  Note that this example uses Langfuse's [V3 (OTel-based)
  SDK](https://langfuse.com/docs/sdk/python/sdk-v3). See their docs for more
  information.
</Callout>

First, configure the Langfuse client [as described by their documentation](https://langfuse.com/docs/opentelemetry/example-python-sdk):

<Snippet
  src={
    snippets.python.opentelemetry_instrumentation.langfuse.client
      .configure_langfuse
  }
/>

Langfuse will set the global tracer provider, so you don't have to do it manually.

Next, create an OpenAI client [using Langfuse's OpenAI wrapper `langfuse.openai` as a drop-in replacement for the default OpenAI](https://langfuse.com/docs/integrations/openai/python/get-started) client:

<Snippet
  src={
    snippets.python.opentelemetry_instrumentation.langfuse.client
      .create_open_ai_client
  }
/>

And that's it! Now you're ready to instrument your Hatchet workers with Langfuse. For example, create a task like this:

<Snippet
  src={snippets.python.opentelemetry_instrumentation.langfuse.worker.task}
/>

And finally, run the task to view the Langfuse traces (cost, usage, etc.) interspersed with Hatchet's traces, in addition to any other traces you may have:

<Snippet
  src={
    snippets.python.opentelemetry_instrumentation.langfuse.trigger.trigger_task
  }
/>

When you run this task, you'll see a trace like this in Langfuse!

<img
  src="/langfuse-trace.png"
  alt="example-langfuse-trace"
  className="max-w-full mx-auto my-8"
/>
